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What is this study about? 

The study examined whether shifting from tradi- 
tional district management to management by 
a for-profit or nonprofit organization improves 
student achievement. 

The study analyzed data on six cohorts of elemen- 
tary and middle school students (2002-03 through 
2007-08) in the Philadelphia school district. 

The study used three approaches to estimate 
effects on student test scores. For two of the three 
approaches, the study did not present evidence on 
test score equivalence at baseline, which is one of 
the standards used by the WWC to assess validity. 
This quick review focuses on the approach that pres- 
ents evidence on baseline test-score equivalence. 

The more rigorous approach contains two main 
analyses. The first analysis (referred to below as the 
“District Comparison Analysis”) examined data from 
six school years and included about 1 16,000 students. 
The math and reading scores of students in schools 
that were managed by a for-profit company or a non- 
profit organization were compared to the test scores 
of students in the 71 lowest-performing schools that 
remained under regular district management. 



What Types of Schools Were Contrasted? 



In the summer of 2002, the Philadelphia school 
district contracted with private organizations 
to take over the management of its 46 lowest- 
performing elementary and middle schools. 

Thirty schools were contracted out to the 
for-profit management organizations Edison 
Schools, Victory Schools, and Chancellor 
Beacon Academies. 

Sixteen schools were contracted out to 
four nonprofits: Foundations, the University 
of Pennsylvania, Temple University, and 
Universal Companies. 

These organizations were subject to the district’s 
existing collective bargaining agreements. 



The second analysis (referred to below as the 
“Matched Comparison Analysis”) matched for- 
profit-managed schools and nonprofit-managed 
schools to the 30 schools under regular district 
management that were most similar to them based 
on prior test scores and student demographic 
composition. The test scores of students in schools 
managed by a for-profit or nonprofit organization 
were compared to those of students in matched 
comparison schools. 



(continued) 
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WWC quick reviews are based on the evidence published in the report cited and rely on effect sizes and significance levels as reported by 
study authors. WWC does not confirm study authors’ findings or contact authors for additional information about the study. 

The WWC rating refers only to the results summarized above and not necessarily to all results presented in the study. 
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WWC Rating of District Comparison Analysis 



The District Comparison 
Analysis described in this 
report is not consistent with 
WWC evidence standards 

Cautions: The analysis compared groups of 
students that were not similar at baseline. The 
students enrolled in schools managed by for-profit 
companies had lower baseline test scores than 
students in the comparison schools run by the 
district. In addition, the student population served 
by the nonprofit-managed schools had a different 
demographic composition from the population 
served by the schools managed by the district. 

What did the study authors report about 
the District Comparison Analysis? 

The study reported that, on average over the six 
years following the shift in school management, 
students in for-profit managed schools had math 
and reading test scores that were not significantly 
different from those of students in comparison 
schools run by the district. During this same period, 
students in nonprofit managed schools had math 
and reading test scores that were about one-seventh 
of a standard deviation lower than students in the 
comparison schools. 

However, the WWC does not consider these results 
to be conclusive because there is evidence that the 
schools were not initially equivalent. The differences 
may reflect differences in the student populations 
served by the different groups of schools and not 
the effect of the school management strategies. 



WWC Rating of Matched Comparison Analysis 



The Matched Comparison 
Analysis described in this 
report is consistent with 
WWC evidence standards 
with reservations 

Strengths: The intervention and comparison schools 
were well-matched on baseline test scores and 
demographic composition of the student body. 

Cautions: Although the study matched treatment 
and comparison schools on a number of observable 
characteristics, unobserved differences between 
the groups of schools and the students that attend 
them may remain. In addition, the study examined six 
years of data. Although the students enrolled in these 
groups of schools may have initially been similar, the 
composition of students enrolled in these schools 
may have changed over time; therefore, the student 
populations in the schools compared in the analysis 
may not have remained similar throughout the 
follow-up period. Differences in achievement between 
the groups of students compared in this analysis may 
not be solely attributable to the effect of different 
school management strategies. 



What did the study authors report about 
the Matched Comparison Analysis? 

The authors found no statistically significant 
differences between student test scores at schools 
managed by for-profit or nonprofit organizations 
and students in their respective matched compari- 
son schools. 

The WWC has reservations about these results 
because students enrolled in privately managed 
schools may differ from students enrolled in tradi- 
tionally managed schools in ways not controlled 
for in the analysis. 



Page 2 







